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INTRODUCTION 



The growth of Hispanic and Indochinese population in the United States 
represents a spectacular rate of increase. The number of Limited English 
Proficient (LEP) students in our schools is increasing at the same time that 
the government agencies are experiencing tremendous budget cuts and, at the 
same time, our society is taking quantum leaps into a new technological era. 
Previous efforts to create sound basis for educational intervention on 
behalf of LEP students have been commendable yet disjointed and uncoordinated. 
Similarly, efforts to provide support to school districts and school district 
personnel have also been disjointed, misdirected and uncoordinated. 

Clearly, what was needed was a new direction, a new generation of effort, 
which draw upon the considerable knowledge we now possess on the problems of 
educational reform, the impediments of institutional change, and the necessary 
process of parent/community involvement and shared decision making. 

With this in mind, Naomi Gray Associates in collaboration with San 
Francisco Unified School District, subcontracting to U. S. Human Resources 
Corporation joined the Department of Education, Office of Bilingual Education 
and Language Minority Affairs in designing, and implementing an "inclusionary" 
process approach to provide support services to local education agencies. We 
say "join" because it was truly a collaborative effort on the part of the 
individuals involved in sharing the experience of designing and implementing 
a new concept in bilingual curriculum/instruction and training materials 
development. 



During Phase I of our 30-month bilingual materials development project 
began in August of 1982, the first major task was to review literature about 
curriculum development and instructional materials which might be available 
in Spanish and English for seventh grade social studies classes. 

Basically, the literature review revealed that very few materials were 
available for this grade level, subject area and language group and that 
those which were available could not be readily used or adapted for local use 
because they did not respond well to the local needs in terms of specific 
content, cultural nuances, or idiomatic use of the first language. 

The next major step/task in the project was to recruit the technical team 
members and finalize our review panel members. The technical team members 
were recruited from among the five Spanish bilingual schools within the San 
Francisco Unified School District and from the bilingual/ESL curriculum 
development section and also an administrator from the bilingual education 
department, so that among the six members of the technical team there existed 
a variety of educational experiences and perspectives. The training sessions 
held for the technical team focused on the development of the process/approach 
model for bilingual curriculum and instructional materials development. At 
this same time a review panel of educational specialists with expertise in 
bilingual education and/or bilingual curriculum and instructional materials 
development was established, which included a member of the bilingual educa- 
tional department of a local university, the head of a local TESOL chapter, 
and a member of a local educational research organization which has extensive 
experience and expertise in bilingual education. 
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After both teams were established then began the process/approach model. 
The project staff held extensive and intensive training sessions for the 
technical team members. The team adjusted the process/approach we originally 
proposed, a new and more flexible design was adapted to fit different situa- 
tions depending on the specific needs and perspectives of the technical team. 

One of the issues which emerged during these discussions was the fact 
that the curriculum developer from the S.F,U.S,D. bilingual education depart- 
ment curriculum development office had developed a bilingual unit on the 
history of California which could be adapted using our process/approach. 
That is. The Story of California had been translated into Spanish and English, 
as well as six other languages, but still needed revisions and was lacking 
curriculum materials to go with it such as a teacher's guide and student 
worksheets. The technical team agreed that these materials would be particu- 
larly responsive to local needs. 

The next step in the process/approach model was to actually develop the 
needed bilingual curriculum and instructional materials. The Technical Team 
commenced this step in Phase I of the project in February, 1983. During this 
time we held regular technical team meetings, usually one and often two per 
week in order to discuss the key issues involved in curriculum development. 
The goals and objectives for cognitive, language, and affective development 
were discussed at great length, since the materials had to be properly 
designed in terms of these dimensions. 

In terms of cognitive development, it was decided that the concepts could 
be higher than the language level because many students had higher cognitive 



development than language development, especially in Spanish. Also,, the 
technical team decided that the curriculum and instructional materials to be 
developed should address the affective development of the students, since 
students in seventh grade are in a state of rapid and often confusing transi- 
tion in adolescent development and are often having difficulty adjusting to 
a new culture. Thus, the ideas in the text and the worksheets and the teach- 
ers' guide should relate as much as possible to the students* native culture. 

During the spring, summer, and fall, 1983, the project worked with the 
Technical Team in developing the bilingual curriculum and instructional 
materials using the process/approach. During this same time we also worked 
developing the evaluation design and field test instrument to be used during 
the field test, which would be conducted during the spring semester of the 
1983-84 school year. This evaluation design included the design and the field 
test instruments, which included feedback from the teachers and students 
about the curriculum and instructional materials as well as a classroom ob- 
servation sheet for classroom observations by the technical team in order to 
secure first hand feedback from field test teachers and to observe the actual 
use of the curriculum and instructional materials in the clarsroom. It was 
also important in the evaluation design to measure student language proficiency 
in Spanish and English, so the field test instrument included the Oral Language 
Proficiency Rating Scale (OPRS). The field test instrument also included 
measures of students' academic performance utilizing the materials, based on 
teacher ratings of student performance. 

The preparations for the field test were also made by contacting school 
districts and teachers in order to obtain participation for the field test. 



Finally, six or possibly seven classes in the San Francisco Unified School 
District had agreed to participate in the field test of the bilingual 
curriculum and instructional materials. These classes are bilingual Spanish- 
English seventh grade social studies classes. The field test will include 
about 210 - 250 students in these classes in Spanish and English. 

On February 10, 1984 a training workshop was conducted for field test 
teachers at Roosevelt Junior High School in Oakland, although field test 
teachers from S.F.U.S.D. were also workshop participants. At this workshop, 
one of the members of the technical team who was a curriculum developer and 
developed The Story of California trained the field test teachers in the 
contents of The Story of California and how to use it during the field test. 
Another member of the technical team trained the workshop participants in 
classroom management. Also, the evaluator who had developed the evaluation 
design trained the teachers in how to use the field test instruments and how 
the evaluation data would be collected. 

During this time we also wrote three manuals (1) a How To Workbook, 
(2) a Training Manual, and (3) a Resource Manual for use by bilingual teach- 
ers, curriculum developers, district administrators, school principals, 
parents, and others who need to develop bilingual curriculum and instruc- 
tional materials. The draft was submitted November 30, 1983 and the revised 
draft was submitted February 8, 1984* As per our contract, each manual/hand- 
book/workbook is 75 - 100 pages in length. These will be resubmitted as 
working drafts in June at the end of the field test phase in Phase 1. 



The primary focus of Phase II has been continual revisions of training 
materials, in particularly, Resource and How to Workbooks, the field testing 
of the instructional materials • The field testing of the "Story of California" 
and the curriculum and instructional materials did encountered some difficul- 
ties. Generally, both the transition of students, and their language level(s) 
makes it difficult to determine/evaluate the students' performance. The 
concern here is that the sample is too small to adequately determine results. 

During this period a great deal of time was spent with bilingual teach- 
ers, bilingual resource specialists, and bilingual student teachers, our 
emphasis here was to determine the effectiveness of the resource and training 
manuals. Their input was incorporated and revisions were made. We continued 
to revise the training manual based on the input we received from the various 
audiences. 

June was a month of major activity, finalizing all aspects of the 
curriculum and instructional materials and completing revisions on the 
Training, Resource and How to Manuals. Establishing new field test site in 
Redwood City as well as finalizing data collection in San Francisco and 
Oakland School districts in order to prepare the field test results report. 

The completion of all major deliverables, which included a theoretical 
justification in the process model, this revision was performed with input 
from San Francisco's Emergent/Resource Specialist. 

Data collection and analysis continued, findings in the Redwood City 
evaluation did not demonstrate sufficient change, however, relevant findings 
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were incorporated into the curriculum and recommendations for changes in the 
evaluation design were incorporated into the evaluation forms. 

All major deliverables were revised, completed and sent to Washington. 
This included the final revisions in the Student Workbook and the Teacher's 
Guide which accompany the Story of California and writing a Teacher's Edition 
to the Student Workbook, this included all the worksheets with correct answers 
filled in (the Teacher's Edition to the Student Workbook represent an example 
of how we incorporated the input received from the teachers). 

It was during this phase that contact was made with the State Department 
of Education, Office of Bilingual Education in which the goals and objectives 
of both the training materials and instructional materials were presented and 
explained. 

During the fall we arranged for another field test in Oakland and San 
Francisco. A new dimension was added to this field test, not only did we col- 
lect data on the students' ora) proficiency, but we also collected data on the 
students' written language proficiency. The "Written Language Proficiency 
Rating Scale" was developed by NGA staff in order to rate the students' written 
proficiency. We collected writing samples from all the students who partici- 
pated in this final field test. 

Other activities during Phase II included training sessions which utilized 
the training and resource manuals, these workshops were conducted in Berkeley, 
Redwood City and Oakland, California. 



The last two quarters in Phase II involved the completion of several 
tasks. Firstly, .continued field testing of the instructional materials, data 
collection and analysis, writing the final evaluation report to be submitted . 
Secondly, scheduling and preparation of training workshops for teachers, 
resource teachers and bilingual curriculum specialists. 

The thrust of Phase II was continual and final revisions of training 
materials, final evaluation of instructional materials and training workshop 
for bilingual school district personnel. Because the field testing was such 
a critical part of our project we have incorporated parts of the final evalua- 
tion report. 
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EVALUATION REPORT 
ON FALL, 1984 FIELD TEST 
OF 

THE STORY OF CALIFORNIA 



One of the major products of this project was The Story of 
California , a middle-school, Spanish-English social studies unit on 
California geography and history. The unit contains a textbook 
called The Story of California, written in English on one page and 
Spanish on the facing page. The Student Workbook contains written 
exercises on each section of each chapter of the textbook. The 
Teacher's Guide contains classroom activities for every section of 
every chapter in the text, as well as scope and sequence guides for 
the entire unit and introductory material for the teacher* 
Field Test 

These materials were field tested in two classrooms at Potrero 
Hill Middle School in the San Francisco Unified School District and five 
at Lazear Elementary School in the Oakland Unified School District, for 
a total of seven classrooms in all. The elementary school was used 
because teachers there felt the students could handle the material. 
Since some middle schools had not wanted to use the materials because 
California history is usually taught in 4th grade, it was decided to 
see how well the materials worked in 4th and 5th grades. 



Oakland 



The classes using the materials were: 

4th grade 



/class A 
Class B 
Class C 
Class D 
V^Class E 



San 

Fransisco 



31 students 

29 students 

30 students 
29 students 
29 students 
23 students 
33 students 



4th grade 
5th grade 
5th and 6th grade 
6th grade 

Class F 6th, 7th, and 8th grade 

Class 6 7th and 8th grade 

204 students 

The field test was conducted in the Fail, 1984 semester. San 
Francisco classes began using the materials when school began in 
September, and the Oakland classes began using the materials in October, 
1984. Evaluation data on the materials was collected until the field 
test was concluded on January 18, 1985, although all classes will 
continue to use the materials until the end of the school year in 
June, 1985. 
Evaluation Materials 

Evaluation data included ratings of students' oral and written 
proficiency in both Spanish and English, student and teacher feedback 
about how they liked the materials, and teacher feedback about student 
performance. During the semester teachers rated student's oral 
proficiency or the Oral Proficiency Rating Scale (OPRS), on which 
students are rated from one to five on their pronunciation, grammar, 
vocabulary, comprehension, and overall communicative skill in Spanish 
and English; the last dimension is used as the overall rating of oral 
proficiency (see OPRS in appendix). 
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The Written Proficiency Rating Scale (WPRS) was devised to rate 
students' written proficiency in Spanish and English. Modeled after 
the OPRS, it asks judges to rate students' one paragraph writing 
samples on a scale of one to four on punctuation, spelling, vocabulary, 
graFumar, and overall effectiveness in communication, with the last 
dimension being the overall rating of written proficiency (see WPRS 
in appendix). 

Teachers were asked to complete a Teacher Feedback Sheet when 
they completed each chapter. This sheet asked them to rate how well 
students learned the language skills, geography skills and key 
concepts taught in each chapter, the appropriateness and effectiveness 
of the worksheets and classroom activities, and their perceptions about 
the textbook. The Student Feedback Sheet asked students how hard the 
materials were and whether they liked the chapter. (See appendix 
for Teacher Feedback Sheet and Student Feedback Sheet.) 

FIELD TEST RESULTS 
One-paragraph writing samples were obtained from most students 
in all seven classes, and the WPRS was completed for each. Six of 
the seven teachers completed the OPRS ratings for all students in their 
classes. For chapter one materials, we received Teacher Feedback Sheets 
and Student Feedback Sheets from all seven classes. For chapter two 
we received Student Feedback Sheets and Teacher Feedback Sheets from 
four classes, although all classes completed chapter two. One teacher 
submitted a Teacher Feedback Sheet and Student Feedback 
Sheet for chapters three and four. Only two other classes 
even got to chapter three during the field test period, since most were 



using the materials to supplement other texts rather than as the main 

text for the unit. 

Student Language Proficiency 

Tables 1 and 2 summarize the students' oral and written 
language proficiency, as gauged by the OPRS and WPRS respectively. The 
entries in these tables represent the number of students in each rating 
category. 

Table 1 shows that students' oral proficiency in English was rough- 
ly normally distributed^ with most students scoring fairly well (3 or 4 
rating) and fewer having very low or very high oral proficiency. In 
Spanish, however, the distribution is bi-modal; that is, the largest 
proportions of students are at the high or low end of the scale. This 
means that students generally can either speak Spanish or not; if they 
can, they can generally speak it well. 

Written proficiency, as indicated in Table 2, shows a different 
profile* Most students were rated as having very low proficiency in 
both English (78%) and Spanish (89%). Only a small fraction of students 
were rated as having high written proficiency in either language. 
Chapter 1 Feedback 

Chapter 1 is about maps, continents, and the U.S. and California 
governments. The class time spent on this chapter ranged from 3 to 
20 days. The teacher and student feedback on Chapter 1 showed that the 

reading level in English was appropriate for a majority of students. 
The teacher feedback sheet showed that it was "about right" for 58% of the 
students in the field test, "too high" for 22%, and "too low" for 20%. 
Table 3 shows that 58% of the students felt it was about right or a little 
easy or hard, while the rest thought it was very easy (35%) or very hard 
(6%). These figures are generally compatible between the teachers and 



12 



sz 



C 



JO 
<T3 



O 

u 



Q. 

o 



2i 

U 
CO 



n3 



o 



o 



</7 



> 



o 



ro 
$- 

> 

o 



c 



<T3 

o 



c: 

UJ 



CM 



o — 



eg 



Osl 



CO ^ 



Lf) 



00 



00 



00 



c 
o 

Ol 



CM 



in 



o 



o 

OsJ 



in 



CD 
CO 



CNJ 
CO 



00 O o> 

CM OO CM 



CM 



CM 



^ LO 



I 



00 
I 



00 

I 

CD 



00 

LO 



oo 



CO 



CM 



LD 
LO 



cn 

LO 



< 

o 



CM 
OO 



CO 



a; 
cr 



T3 
C 
(O 

O) 

o 
to 

<T3 



x: fo 
c/i <D 

— • CL 

c: 00 

CO 

o JZ 
c: 00 

"D C 
C 

It} CL 
LO 

o ^ 

—I to 00 

Cf> clc 

03 CO <T3 
H- CX 

4JLO 

o ^ 

Q. O <y 

CO CO 
4-> 4-> 4-> 

ceo 

0) <u c 

=3 =3 O 
4-> 4-> ^ 

^ OCO 



13 



ERIC 



2! 

O 

CO 



CO 

a: 

Q. 

3: 



CO 



(/^ 
a; 
c 

> 



> 

o 



o. 



CM 



o ^ 



CM 



OS 

't; >> 

-Q O 

o 

Q. 



o 
c 



o 



CO 



c 



CM 



o 



00 



2 



05 

c 



a; 
■o 
to 
(. 

CD 

to 
to 



-79^ 



■35 *3 t?r 



o o o o 



CM ^ ^ O LO ^ 



vo CM v£> vo 



CO 00 ^ in o CM 



CM 



o 



CM 
CO 



O 



CM 



in 

CO 



tn 



CO 



CM 



in 



CM a\ 

CO ^ 



in 



CM 



O 
CM 



GO 



O 

in 



CO CM in ^ Lo vo 



CO 
CM 



o 

CO 



CM 



CM 



CM 



O 
CO 



GO 



o 
o 



CO 



in 



vo 
I 

in 



vo 



GO 

I 

I 

vo 



00 



LL. <D 



o 



14 



<v 
+■> 
to 

4-> 

CO 



< 

S , 

L±J ' 
^ Ll. 



o 



o 



03 

o 



o. 





N/A 


1 


1 




1 


1 


I 




OsJ 






>> 

to 

n3 IT) 

, 9 . 




CO 






CM 


00 


00 


LO 
CO 




:deas 






00 


0 


KO 


CM 


00 




CO 
LO 


en 

CM 


c 

CO 


CO 


00 


0 


CM 


CM 




vo 


00 


CO 


CO 


CO 


C\J 


C\J 




— 


CM 




CO 




CO 
CM 


ro 






0 




0 


0 




LO 


0 








N/A 




1 


iO 


LO 


1 


vo 


1 


to 


LO 
CM 




>> 

<t3 in 




KD 


in 




LO 






CTi 

*^ 


CM 








0 


0 




LO 








0 


Spani! 


CO 






LD 


CM 






CO 


CM 
CO 


GO 






0 


t\ 


— 


CM 


LO 


CM 


CO 


0 

CM 








CsJ 






0 


LO 


0 


0 










1 


1 




1 


1 


1 




CM 




























>> 

t/) 


CO 


LT) 




<T» 






0 


LO 

vo 


LO 
00 


ish 








00 




00 


0 




U3 
CO 


0 

CM 






















Engl: 


CO 


CO 




LO 




CO 


to 


LO 


LO 


LO 
OJ 




C\J 


C\J 


o% 


0 






CO 




CVi 


CO 




•a 

u 
to 




CM 


0 






CM 






U5 






00 


to 


00 
CM 


<M 


vo 
CM 


0 

CM 


CO 


CO 
GO 


0 


CLASS/ 
GRADE 


A/4th 


B/4th 


C/5th 


D/5-6th 


E/6th 


F/6-7-8th 


G/7-8th 


TOTAL 





15 



students, although students tended to rate the materials as easier than 
the teachers. Given the diversity in grade level and language abilities 
in these classes, the reading level covers a wide spread of students • 

Similar figures occur for the reading level in Spanish* Teachers 
said the level in Spanish was about right for 60% of the students, too 
high for 32%, and too low for 8%. Thirty-nine percent of the students . 
said the level was about right, 9% said way too hard, and 27% way too 
easy; 25% said the question was inappropriate, meaning they could not 
read Spanish at all. It is also interesting that teachers said they 
speak in English about 75% of the time, while students speak in English 
about 80% of the time. 

The teachers' ratings of student performance reflect the language 
diversity found in the classes. The following figures show the percent- 
age of students in each category (rounded to the nearest 5%). 



Chapter 1 : Average Student Performance 





Extremely 

Well 
(85-100%) 


Satis- 
factory 
(70-84%) 


Unsatis- 
factory 
(40-69%) 


Poor 
(0-39%) 


Spanish Vocabulary 


40% 


30% 


20% 


10% 


English Vocabulary 


65% 


20% 


12% 


3% 


Main Concepts 


25% 


40% 


20% 


15% 


Map Reading Skills 


15% 


50% 


20% 


15% 


Graph/Chart Reading Skills 


12% 


50% 


20% 


15% 



These data show that 65%-85% of the students are learning a satisfactory 
amount with better performance in English language areas than Spanish or 
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geography areas, where only 65% or 70% performed satisfactorily. There 
appears to be a problem with learning when 35% of the students are not 
learning the material at a satisfactory level and 15% showed poor 
learning. Indeed, Table 3 shows that 10% of the students rated the 
main ideas as hard (1 or 2 on a scale of 1 to 5). 

One key to unsatisfactory performance in geography skills could be 

the interest of the chapter for the students. Table 3, item 4 shows 
that 27% of the students rate the chapter on the boring (vs. interesting) 
side of a 1 - 4 scale. On the Teacher Feedback Sheet, item 14, two 
teachers said the chapter was boring to students (1-3 rating on a scale 
of 1 - 6), and three more rated it a 4 out of 6, which reflects moderate 
interest at best. There was a consensus that the sections on the law- 
makers and the map directions were particularly uninteresting to students. 
In addition, teachers suggested in item 13 that the graphics were too 
small and/or too complicated. Thus, chapter 1 needs to be revised to 
be made more interesting; this, in turn, would help improve student 
performance by motivating students to learn more. 

Another probable reason for the students' poor ptrformance in 
geography is the fact that few teachers did any of the classroom activi- 
ties for chapter 1. The number of teachers doing the classroom activities 
are: 

Topic Did it Did Not Do it 

Glomes 3 4 

Continents 2 5 

Maps-Directions 1 6 

Map -Directions 1 6 

Distance on a Map 3 4 

Continents, Maps 0 7 
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Those who did do an activity almost always rated it as appropriate 
in level of difficulty, interesting to themselves and students, and as 
working well in class. These activities, available in the Teacher's 
Guide, were designed to reinforce the concepts and skills introduced in 
text' and the worksheets* Thus, by not using the classroom activities,- 
teachers seldom gave students the chance to consolidate whatever they 
did learn from the text or worksheets. 

The student worksheets are another major part of the unit. Feedback 
on these was generally positive. There were five worksheets with a total 
of 18 sections- Fourteen of these were rated by a majority of the teach- 
ers (and usually all teachers) as being '*about right" in difficulty and 
on the ''effective" side of a 1-6 rating scale. The other four sections 
were rated as "too hard" by a majority of the teachers: 

1) Worksheet 2, Section III, on the relationship between a drawing 
of land and a map of that land; 

2) Worksheet 4, Section III, on measuring distance on a map; 

3) Worksheet 5, Section I, on making a bar graph; 

4) Worksheet 5, Section IV, on taking information from a bar graph 
and inserting it into a pre-cut pie graph. 

The rating of these sections —containing map and graph exercises — 

as too hard corresponds with the teacher ratings that 35% of the students 
did not learn map and graph skills at a satisfactory level. 

The other items on the Teacher Feedback Sheet covered a variety of 
miscellaneous issues. There was no consensus about strong and weak points 
but on suggested changes, two teachers said the book needed a better ex- 
planation of the lawmakers and the government and a better explanation 
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for measuring distances on a map. All teachers said they would use this 
chapter again with their class. 

Four of six respondents liked the language format of English on one 
page and Spanish on the facing page. They said it gave the students the 
opportunity to read and learn in the language most comfortable to them. 
They said most students liked it too. especially Spanish-speakers with 
very limited English skills, because it gave them the opportunity to 
learn the English but made them feel secure by also having the correspond- 
ing Spanish. 
Chapter 2 Feedback 

Chapter 2 is about the various land forms in California. We 
received Teacher Feedback Sheets and Student Feedback Sheets from 

three classes. One other class submitted only Student Feedback Sheets 
while another submitted only a Teacher Feedback Sheet. 

The variation in time spent on the chapter continued in chapter 2. 
The four classes reporting spent 6» 4, 20, and 15 days respectively on 
this chapter. Clearly the coverage in the first two classes (4th and 
5 - 6th grades) was minimal, using The Story of California only to 
supplement other materials. The latter two classes were in middle 
school and used the materials as the main text. 

Teachers and students both spoke in English about 85% of the time. 
The reading level in Chapter 2 was "about right" for most (about 60%) 
of the students but ''too high" for most of the remainder, whether in 
English or Spanish. 

Student performance figures show that a high percentage of students 
still are not learning the material well: 
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Chapter 2: Average Student Performance 



Extremely Satis- Unsatis- Poor 

Well factory factory 

(85 - 100%) (70 - 84%) (40 - 69%) (0 - 39%) 



Main concepts . 15% 

Map reading 10 

Graph/chart 25 
reading 

Research 30 

Spanish vocabulary 15 

English vocabulary 25 

Spanish grammar 10% 



50% 

45 

35 

35 
60 
50 
30% 



30% 

30 

30 

25 
20 
20 
50% 



5% 
15 
10 

10 
5 
5 

10% 



Three factors probably contributed to the poor performance by 
students. One was that most classes did not spend enough time on the 
chapter for students to learn the material well. Two of the four teach- 
ers reporting only spent four or six days on the chapter; interviews 
showed that non-reporting teachers spent similar amounts of time on the 
chapter. This Is simply not enough time to learn the material. 

A second reason is the relatively low interest the material had for 
the students, which undoubtedly led to low motivation to learn. All 
reporting teachers said the materials did not hold great interest for 
the students, as reflected by ratings of four or less on a scale of 1-6. 
Thirty four percent of the students rated the materials as boring (see 
Table 4). Most teachers said that graphics were not particularly in- 
terestina to students either. 
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A third reason is that as with chapter 1, the teachers did not 
make sufficient use of the classroom activities and thus did not 
reinforce what was learned in the text and worksheets. Three of the 
four reporting teachers did not use any classroom activities from the 
Teacher's Guide, and the remaining teacher used four of the six acti- • 
vlties in this chapter. That teacher had problems with these activities. 
Two called for the students to work in small groups, but that class 
was fairly rowdy and the small groups did not work well. The other 
two activities focused on speaking complete sentences in Spanish; 
however, the teacher felt the language was inappropriate and found 
the activities worked well in English. 

Chapter 2 contained 6 worksheets with a total of 21 sections on 
California geography. A majority of teachers rated 15 of the 21 sections 
as being "about right" in difficulty and "effective" (i.e., 4 or more 
on a scale of 1 - 6). Three sections were rated by a majority of 
teachers as "too hard" (Worksheet 9, section I; Worksheet 10 » sections 
I, and II). In all three cases the section was in Spanish and the 
comments said that "many Spanish speaking students can't read Spanish." 
This problem is consistent with the data on Students' Spanish language 
proficiency— i.e., high or no oral proficiency in Spanish and low 
written proficiency. However, this problem was not expected when the 
materials were being designed. On three worksheet sections there was 
a split judgment. Worksheet II, Section III required students to 
alphabetize words in Spanish. Two teachers said it was "too hard" 
because it was in Spanish and two said it was "too easy" because it 
was in Spanish! The other two sections had a rating of "too hard" by 
two teachers and "about right" by two teachers. Both sections were 
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f ill-in-the-blank exercises which did not have a selection of answers 
in a sidebar to choose from; both teachers who felt this exercise was 
too hard said that it needed to have the answers available. 

The final item on the Chapter 2 Teacher Feedback Sheet did not 
generally elicit a consensus from the teachers. Two 
of the four teachers did say the chapter was too general 
and did not provide enough detail on each type of land form (i.e., 
deserts, mountains, etc.)* 
Feedback on Chapters 3 and 4 

Since only one class submitted a Teacher Feedback Sheet and 
Student Feedback Sheet for chapter 3 and 4, the detailed analysis pre- 
sented above for chapters 1 and 2 is unnecessary. It is interesting to 
note that this class has the oldest students (Class 6, 7th and 8th 
graders). For chapter 3, on The First People in California,- the patterns 
of student performance and teacher and student reaction to the mate- 
rials are similar to chapters 1 and 2. A little over half the students 
rated the material as easy but boring (see Table 5). The teacher 
strongly attacked the chapter on cultural grounds, stating that it over- 
simplified the Indian culture by making it appear too idyllic and not 
portraying the activities which men and women did in common and by not 
explaining that both men and women fished, built houses, made tools, etc. 

Of the 18 worksheet sections rated, 10 were rated as about right 
in difficulty and effective. Three sections— all on alphabetizing — 
were rated as too easy. Five sections were rated as too hard, four 
because they were in Spanish and one because it was a f i 11-in-the-blank 
exercise without possible answers listed (i.e. , same criticism as 
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in chapter 2)* 

Two of five classroom activities were done from chapter 3. One 
was rated as acceptable in difficulty, as only moderate in interest for 
the students, and as working only moderately well in class. However, 
this activity was not done as designed; it included a set of pictures 
depicting the food chain which groups of students were supposed to put in 
order. In the class, the pictures were only used as a basis for class 
discussion. The other activity was an alphabetizing activity in English; 
the teacher said this activity worked fairly well in class. 

Chapter 4 is on the first Europeans in the new world— i.e., 
Columbus in the new world, Cortez in Mexico, and the first Spanish 
activities in California. The patterns of student performance were 
similar to the data on the first three chapters.' However, while the 
teacher felt the text was interesting to the students, 11 of 23 students 
rated it as boring. 

Eleven of the seventeen worksheet sections used were rated as 
effective and about right in difficulty. The other 6 sections were rated as 
too hard. Three were in Spanish — two requiring students to write 
sentences and one to fill in a blank to complete a sentence. Three were 
in English — two requiring students to write a short answer to a 
question of fact and one to fill in the blank to complete a sentence. 
All of these exercises have a common theme; namely, they are exercises 
to test students' comprehension of the content without referring them 
to the text or providing choices of answers. 

The teacher did do all five chapter 4 activities in the Teacher's 
Guide. Two activities were in Spanish and required small groups of students 
to work together. The teacher said that both activities were too hard 
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because they were in the wrong language for students' abilities; 
another comment was: "Any activity that involves autonomous small 
group activity meets with resistance and disorder in my class." 

The three English language activities were about right in language 
difficulty and fairly interesting to the teacher and students. Two were 
about right in cognitive difficulty and worked reasonably well in class, 
while the third (a role--playing activity about slavery) was too 
difficult for the students and thus worked poorly in class; the 
teacher used it as the basis for class discussion rather than doing 
role-playing with the students* 
Conclusions 

The field test data suggest several important conclusions about 
these materials. Some pertain to the assumptions behind them, some 
pertain to how to revise the materials, and some pertain to teacher 
training on the use of the materials. 

One of the key assumptions behind these materials was that students' 
Spanish language abilities are better than their English proficiency, 
which is limited. However, this appears to be true for oral proficiency 
only and not for written proficiency. In reading and writing many of 
these students are as limited or even more limited in Spanish than En- 
glish (see Tables 1 and 2). This means that written Spanish cannot be 
counted on to be the basis for written English language development, as 
originally assumed. Most cannot read complex Spanish and prefer to read 
or learn in English anyway. Thus, many difficult parts of the written 
exercise in the Teacher's Guide and Student Worksheets should be trans- 
lated into English. 
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A second key finding was that materials could be used by 4th or^ 
5th graders, as well as middle school classes. This is true because 
the English and Spanish language proficiency of these students from 
4th through 8th grade is not significantly different either orally or 
in writing. There were some activities which were cognitively too 
hard for 4th graders, but not linguistically too hard, (The reasons 
why the older children did not have better language proficiency than 
the younger ones are a complex and separate issue.) 

This finding has important implications for the applicability of 
the materials. California history is typically taught in 4th grade. 
This unit was designed for middle school students in San Francisco 
who were newly arrived from Mexico, Central and South America and who 
could benefit from learning about California history before studying 
world history later in middle school or in high school. Thus, the 
unit was designed for an unusual population and was not appropriate for 
most middle school students because they have already taken California 
history in 4th grade. The result was that it was difficult to find 
middle schools that wanted to use the materials, especially outside 
San Francisco. However, the finding that the materials can be used 
successfully by 4th graders greatly expands their applicability to the 
more common setting of a 4th grade California history unit, as well as 
being useable with recent immigrants to middle schools. 

The field test results for chapters 1-4 suggest some revisions 
needed in the materials themselves, most notably: 

1) the text needs to be made more interesting; both teachers and 
students rated the text as fairly uninteresting and graphics 
as not very appealing. 
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2) chapter 3 The First People — should possibly be revised to 
better reflect the complexity of life among the indigenous 
people, although these perceptions only came from one teacher; 

3) the written Spanish should possibly be downgraded a little in 
difficulty, because it is now harder in Spanish than English 
but students' written proficiency is about the same in both 
languages; however, more consultation with field test teachers 
and other potential users is needed before such drastic revi- 
sions are undertaken; 

4) revise the student worksheet sections which are too hard, 
especially by translating the Spanish sections into English 
and simplifying the open-ended-question exercises; the sections 
rated "too easy" should be left intact for the students with 
very limited language skills; 

5) translate some of the Spanish classroom activities, especially 
the writing activities, into English where they are too hard 
for the students in Spanish; 

6) consider revising some of the small group activities into 
independent activities, since classes can tend to be too dis- 
orderly during small group activities. 

The field test results also suggest three ways in which the in- 
service training of teachers who will use The Story of California can 
be improved. First, the training should strongly stress the importance 
of the classroom activities and how to do them. Few field test teachers 
used very many activities, and this may have been an important factor 
underlying the unsatisfactory performance of a third or more of the 
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students, since the activities were designed to reinforce the concepts 
and skills presented in the other materials. 

Second, the teacher training should provide more definite guide-^ 
lines on how long to spend on the material. It was designed as a 
semester long unit in which it is the primary text. Using these as 
supplementary materials and only spending 3-4 days on a chapter is not 
an efficient or effective use of the materials. Classes should proba- 
bly spend 7-10 days per chapter on these materials, assuming that most 
reading is done outside of class. 

Third, the teacher training should focus on getting teachers to 
use the scope and sequence charts in the Teacher's Guide as part of 
their lesson planning process. Teachers all say they do not have 
enough prep time, but few used the scope and sequence charts, which were 
designed to help in planning. Thus, the training should emphasize how 
to understand and use these charts, thereby helping to increase teachers' 
understanding of the goals of the materials, encourage the full use of 
all the materials, and improve teachers' lesson planning all at the 
same time. 
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Appendix 



Oral Proficiency Rating Scale (OPRS) 

Written Proficiency Rating Scale (WPRS) 

Teacher Feedback Sheet (sample 
for chapter 1 ) 

Student Feedback Sheet (sample 
for chapter 1 ) 
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RECOMMENDATIONS 



One of the objectives listed in our proposal to U. S. Department of 
Education, Office of Bilingual Education and Minority Language Affairs was to 
compile a list of recommendations which would be helpful to the department. 
The recommendations are based on the entire project and our experience, as 
follows: 

1) in curriculum development project, teachers need to be released 
from regular classroom duties in order to attend intensive training 
sessions specifically designed to enable them to become developers. 

2) two writers must be assigned to the curriculum development team. 

3) field testing schedule must allow for start-up time, include suffi- 
cient time. 

4) field test must be in accordance with school year. 

5) if training materials are developed, they should be made available 
to school districts. 

6) training bilingual teachers and administrators in order to enable 
them to function more effectively and efficiently, so often they are 
overwhelmed by the demands of their job - needs are too great. 

7) contractors meeting should be on-site and limited to three times a 
year. 

8) establish technical -team concept at OBEMLA, and last, 

9) dissemination of training materials. 
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